A video compression approach using super resolution was proposed in a first prior art approach. In the first prior art approach, the spatial size of the input video is reduced to a certain predetermined low resolution (LR) size before encoding. After the low resolution video is received at the decoder side, the low resolution video is up-scaled to the original size using a super resolution method along with some side information (metadata) transmitted with the bitstream. The metadata includes a block-based segmentation of frames where each block is labeled as moving, non-moving flat, and non-moving textured. Non-moving flat blocks are up-scaled by spatial interpolation. For moving blocks, motion vectors are sent to the receiver where a super resolution technique is applied in order to recover sub-pixel information. For non-moving textured blocks, a jittered down-sampling strategy is used wherein four complementary down-sampling grids are applied in rotating order.
However, the aforementioned first prior art approach disadvantageously does not use a smart sampling strategy for moving regions. Rather, the first prior art approach relies on the presence of sub-pixel motion between the low resolution frames in order to obtain super resolution. However, sub-pixel motion is not always guaranteed.
In a second prior art approach, a camera is mechanically moved in sub-pixel shifts between frame captures. The goal is to capture low resolution video which is better suited for subsequent super resolution. For static backgrounds, the method of the second prior art approach is analogous to the jittered sampling idea in aforementioned first prior art approach. However, a fixed jitter is not an effective strategy for the case of non-static backgrounds which is likely in our targeted application, namely, down-sampling high resolution video for subsequent super resolution.